The present invention relates to natural language processing, and more particularly, to synchronization of natural language input and graphical user input in a graphical user interface.
Graphic user interface (GUI) applications perform well in most tasks, in part, because the GUI provides options which relate directly to functionality available within an application and from which the user may select what he/she wants to do. However, GUI applications can sometimes be difficult to use. For example, in some instances, the functionality requires opening many sub-dialog boxes, which may not be intuitive for a user. Additionally, many steps may be required to fill in the GUI elements with the proper information. GUI elements may include pull down or pop up menus, check boxes, radio buttons, text or other types of data input fields, and so on. Moreover, the user may not know where to find available options within the GUI. With small devices (such as hand-held devices, personal digital assistants, wireless phones with networking capabilities, and so on), entry of the information into the GUI may be difficult. Finally, handicapped individuals may have difficult navigating and completing a GUI form, for example.
To overcome some of the limitations associated with GUIs, natural language applications have been developed, which can allow a user to easily express concepts that would be fairly complex in a GUI. For example, in natural language, a user may request “Find all emails to Bill that I sent last week”, “Change the screen resolution to 1280×1024”, “where is the web page about Microsoft Corporation”, “Please delete all unread emails”, and so on. Each such utterance is readily understandable, but may add multiple layers of complexity in completing a GUI form.
However, there are some problems with NL applications, such as discoverability, confusion, dead-end, and training problems. Discoverability refers to situations where the user is unaware of what he/she can do in the program. Confusion refers to problems that arise when users do not understand how the application interpreted a particular text input. A dead-end refers to a situation where the software program provides no option or mechanism for the user to alter what he/she intended. Finally, there is no mechanism to train the user how to utilize NL input.
There is ongoing need for intuitive user interfaces for improved user-program interaction. Embodiments of the present invention provide solutions to these and other problems and provide advantages over existing semantic authoring tools.